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ABSTRACT 


In  response  to  technological  advances.  Network  Centric 
Warfare  (NCW)  emerged  as  a  theory  to  leverage  the 
technology  available  in  today's  world.  Advocates  of  NCW 
claim  that  technology  will  improve  information  sharing  by 
"...robustly  networking  a  force",  thereby  improving  mission 
effectiveness.  This  study  proposes  a  methodology  with 
which  to  test  the  first  tenet  of  NCW:  a  robustly  networked 
force  improves  information  sharing. 

Lessons  learned  from  Human  Systems  Integration  (HSI) 
demonstrate  that  in  order  to  improve  mission  effectiveness, 
characteristics  of  both  the  human  and  the  technology  must 
be  considered.  As  such,  the  impact  of  human 
characterist ics  and  traits  on  mission  effectiveness,  as 
measured  by  individual  and  team  performance,  are  assessed 
using  a  computer  simulation,  C3Fire  . 

Results  at  the  individual  level  suggest  that  persons 
scoring  high  on  extraversion  and  low  on  pessimism  perform 
better  than  those  scoring  low  on  extraversion  and  high  on 
pessimism.  In  contrast,  at  the  team  level,  homogenous 
teams  as  measured  by  optimism-pessimism  perform  worse  than 
diverse  teams.  Results  of  this  thesis  provide  a 
methodology  with  which  to  examine  NCW' s  claims  in  a 
laboratory  setting.  Preliminary  evidence  demonstrates  the 
need  to  consider  human  characteristics  and  traits  in  the 
design  and  composition  of  network  teams. 
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EXECUTIVE  SUMMARY 


Societal  progression  from  the  Industrial  to  the 
Information  Age  has  produced  the  need  for  transformation  in 
warfighting.  In  response  to  technological  advances. 

Network  Centric  Warfare  (NCW)  emerged  as  a  theory  to 
leverage  the  technology  available  in  today's  world. 
Specifically,  advocates  of  NCW  claim  that  technology  will 
improve  information  sharing  by  "...robustly  networking  a 
force",  thereby  improving  mission  effectiveness.  However, 
just  as  with  any  new  theory,  a  methodology  and  procedure 
must  be  in  place  for  validating  its  claims.  This  study 
addresses  this  gap  by  proposing  a  methodology  with  which  to 
test  the  first  tenet  of  NCW:  a  robustly  networked  force 
improves  information  sharing. 

Lessons  learned  from  Human  Systems  Integration  (HSI) 
demonstrate  that  in  order  to  improve  mission  effectiveness, 
characteristics  of  both  the  human  and  the  technology  must 
be  considered.  As  such,  the  impact  of  human 
characteristics  and  traits  (i.e.,  personality,  cultural 
adaptability,  and  optimism-pessimism)  on  mission 
effectiveness,  as  measured  by  individual  and  team 
performance,  are  assessed  using  a  computer  simulation, 
C3Fire . 

Results  at  the  individual  level  suggest  that 
participants  scoring  high  on  extraversion  and  low  on 
pessimism  perform  better  than  those  scoring  low  on 
extraversion  and  high  on  pessimism.  In  contrast,  at  the 
team  level,  homogenous  teams  as  measured  by  optimism- 
pessimism,  perform  worse  than  diverse  teams.  Results  of 


xv 


this  thesis  provide  a  methodology  in  which  to  examine  NCW' s 
claims  in  a  laboratory  setting.  Findings  from  this  study 
demonstrate  the  need  to  consider  human  characteristics  and 
traits  in  the  design  and  composition  of  network  teams. 


xvi 


I.  INTRODUCTION 

A .  PROBLEM  STATEMENT 

Fundamental  changes  in  technology,  weapon  systems,  and 
marketplace  commodities  functioning  as  tools  of  war  are 
affecting  the  very  character  of  warfare.  In  response  to 
these  changes,  the  United  States  Department  of  Defense 
(DoD)  is  transforming.  This  transformation  encompasses  how 
technology  and  new  operational  concepts  and  structures  can 
be  used  to  augment  military  preparedness  (Garstka,  2003) . 

In  addition,  this  recent  transformation  of  the  DoD  has 
embraced  Network  Centric  Warfare  (NCW)  as  an  emerging 
theory  of  warfare  (Cebrowski,  2002) . 

In  theory,  NCW  accelerates  the  ability  to  respond  to 
uncertainty  in  dynamic  situations  by  relying  on  human 
networks.  Advocates  of  NCW  claim  that  adopting  NCW  will 
improve  information  exchange  and  sensemaking  which  are 
referred  to  as  macrocognit ive  functions.  In  theory, 
effectiveness  in  macrocognit ive  functions  increases  because 
complex  cognitive  systems  would  enable  people  to  coordinate 
more  efficiently  across  time,  space,  and  organizational 
boundaries.  However,  claims  of  the  benefits  of  NCW  have 
not  been  systematically  verified  in  field  or  laboratory 
settings . 

One  approach  to  verifying  improvements  in  information 
exchange  and  sensemaking  is  the  study  of  macrocognition. 
Macrocognition  describes  mental  activities  that  must  be 
successfully  accomplished  to  perform  a  task  or  achieve  a 
goal  (Klein  et  al . ,  2003)  .  Macrocognition  is  a  term  used 
to  indicate  the  level  of  description  of  the  cognitive 
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functions  that  are  performed  in  natural  versus  artificial 
laboratory  settings  (Cacciabue  &  Hollnagel,  1995).  As  part 
of  macrocognit ive  functions,  problem  detection,  uncertainty 
management,  and  situated  cognition  occurs.  These  processes 
are  generally  performed  in  collaboration  by  a  team  working 
in  a  natural  setting. 

In  theory,  NCW  is  one  step  towards  military 
transformation.  However,  a  methodology  for  assessing  NCW's 
benefits  in  macrocognit ive  performance  has  not  been 
established.  In  some  cases,  researchers  have  chosen  to 
explain  variability  in  performance  through  individual 
dispositions  such  as  personality.  Therefore,  other  factors 
such  as  individual  characterist ics  and  traits  may  also 
affect  performance. 

B.  OBJECTIVES 

The  objective  of  this  research  effort  is  to  design  and 
establish  a  laboratory  and  research  method  focused  on 
evaluating  the  claims  of  NCW  from  a  human  systems 
integration  (HSI)  perspective.  The  specific  goals  of  this 
effort  include: 

•  To  develop  and  implement  a  laboratory  for 
evaluating  team  performance  . 

•  To  design  and  test  a  set  of  procedures  for 
applying  individual  and  team  performance 
measures . 

•  To  evaluate  the  tenets  and  claims  of  NCW  from  a 
HSI  perspective. 

•  To  demonstrate  the  utility  of  the  Dynamic  Model 
of  Situated  Cognition  (Miller  &  Shattuck,  2005) 
in  providing  a  descriptive  analysis  in  team 
effectiveness  and  its  applicability  in  future 
research . 
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C .  RESEARCH  QUESTIONS 

The  specific  research  questions  addressed  in  this 
study  include : 

•  Are  there  individual  characteristics  and  traits 
that  relate  to  individual  performance? 

•  Are  there  individual  characteristics  and  traits 
that  relate  to  overall  team  performance? 

•  Do  diverse  teams  perform  differently  from 
homogenous  teams? 

•  Are  there  performance  differences  when  blocking 
on  various  team  member  traits  in  team  design? 


D .  BACKGROUND 

As  cited  in  Office  of  Force  Transformation  (2005), 
former  Secretary  of  Defense  Paul  Wolfowitz  (2001)  stated: 

Throughout  history,  warfare  has  assumed  the 
characterist ics  of  its  age  and  the  technology  of 
its  age.  Today  we  see  this  trend  continuing  as  we 
move  from  the  Industrial  Age  warfare  with  its 
emphasis  on  mass  to  Information  Age  warfare, 
which  highlights  the  power  of  networked 
distributed  forces,  p.7 

The  characteristics  of  a  networked  distributed  force 
are  particularly  evident  in  coalition  command  and  control 
(C2)  operations.  Coalition  C2  staffs  are  expected  to  use 
networks  to  share  information  among  diverse  members.  C2 
staffs  involve  multinational  team  members  characterized  by 
divergent  national  interests.  Differences  in  culture, 
organizational  affiliation,  and  professional  backgrounds 
are  believed  to  affect  macrocognit ive  functions  and 
processes  such  as  uncertainty  management  (Klein,  Pongonis  & 
Klein,  2000) . 
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To  capitalize  on  the  perceived  advantage  of  networked 
distributed  teams,  the  U.S.  military  has  adopted  NCW  as  an 
emergent  theory.  However,  with  any  new  theory  there  is  the 
challenge  of  demonstrating  its  utility  and  worth.  If  NCW 
is  to  be  fully  embraced,  empirical  and  qualitative  research 
must  provide  support  for  its  claims. 

Proponents  of  NCW  would  argue  that  NCW  is  required  for 
effective  military  transformation.  In  theory,  mission 
effectiveness  is  enhanced  when  information  sharing  occurs 
as  a  result  of  "robustly  networking  a  force."  As  part  of 
networking  a  force,  Alberts  and  Hayes  (2003)  emphasize 
interoperability,  which  is  the  ability  to  work  together. 
However,  our  understanding  of  technology  and  its 
integration  into  military  operations  to  enhance  mission 
effectiveness  is  incomplete.  A  portion  of  this  gap  between 
leveraging  technology  to  enhance  mission  effectiveness 
includes  the  dynamic  interaction  of  team  members  and  an 
understanding  of  how  individual  characteristics  and  traits 
contribute  to  team  interaction. 

Decades  of  team  research  have  been  conducted,  but  the 
findings  do  not  provide  the  evidence  needed  to  accept  or 
refute  NCW  claims.  Research  at  the  macrocognit ive  level  is 
needed  to  test  NCW  assumptions.  Salas  et  al .  (2003) 

reviewed  twenty-five  years  of  research  on  team 
effectiveness.  They  identified  seven  areas  needing 
research.  Two  of  these  are:  (1)  the  need  to  understand 
distributed  teamwork,  and  (2)  the  need  to  focus  on  team 
culture . 
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E.  HUMAN  SYSTEMS  INTEGRATION 

In  this  thesis,  we  investigate  the  claims  of  NCW  from 
an  HSI  perspective.  HSI  is  a  multidisciplinary  field  with 
eight  basic  areas  or  domains  of  study: 

•  Manpower 

•  Personnel 

•  Training 

•  Human  Factors  Engineering 

•  Health  Hazard 

•  System  Safety 

•  Personnel  Survivability 

•  Habitability 

We  focus  on  three  of  these  eight  domains:  personnel, 
training,  and  human  factors  engineering  (HFE) .  Personnel 
and  training  include  aspects  of  selection  and 
classification;  physical,  cognitive,  and  educational 
characteristics;  knowledge,  skills,  and  abilities;  and, 
finally,  simulation  and  virtual  environments.  HFE  is  the 
integration  of  physical  and  mental  limits,  biases,  and 
behaviors  into  system  definition,  design,  development,  and 
evaluation  to  optimize  human-machine  performance  (Lockett  & 
Powers,  2003)  .  The  personnel,  training,  and  HFE  domains  of 
HSI  provide  a  foundation  by  which  claims  of  NCW  can  be 
tested . 


5 


F .  THESIS  ORGANIZATION 

In  the  next  chapter,  NCW  theory  and  literature 
pertaining  to  factors  that  influence  macrocognit ive 
functions  are  described.  Since  a  methodology  for  testing 
NCW's  claims  has  not  been  previously  created.  Chapter  III 
provides  detail  regarding  hardware/software  use  in  the 
laboratory  setting.  This  is  followed  by  a  description  of 
the  pilot  study  in  Chapter  IV.  Finally,  Chapters  V  and  VI 
present  the  results,  conclusions,  and  recommendations  from 
the  study. 
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D.  LITERATURE  REVIEW 


The  literature  review  is  divided  into  multiple 
sections.  The  first  section  addresses  the  theory  and 
claims  of  NCW.  Other  sections  focus  on  personality, 
cultural  adaptability,  and  optimism-pessimism.  Finally,  a 
conceptual  model  is  described  to  demonstrate  how 
personality,  cultural  adaptability,  and  optimism-pessimism 
influence  how  teams  perform. 

A .  NETWORK -CENTRIC  WARFARE 

NCW  has  become  synonymous  with  recent  attempts  by  the 
DoD  and  other  military  forces  to  leverage  the  networking 
capabilities  of  the  information  age.  Cebrowski  and  Garstka 
(1998),  in  one  of  the  earliest  references  to  NCW,  state 
that  "...NCW  derives  its  power  from  the  strong  networking 
of  a  geographically  dispersed  force."  NCW  is  about  human 
behavior  within  a  network  environment.  The  word  "network" 
(i.e.,  as  a  noun)  refers  to  the  information  technology  and 
can  only  be  an  enabler.  Used  as  a  verb,  "to  network" 
implies  human  behavior,  the  action  and  the  main  focus  of 
NCW  (Office  of  Force  Transformation,  2005) . 

Descriptions  of  NCW  concepts  are  found  in  Power  to  the 
Edge  (Alberts  &  Hayes,  2003)  .  Stemming  from  the  work  of 
others,  they  list  four  tenets  that  describe  the  U.S. 
military's  approach  to  NCW: 

•  A  robustly  networked  force  improves  information 
sharing . 

•  Information  sharing  and  collaboration  enhances 
the  quality  of  information  and  shared  situational 
awareness . 
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•  Shared  situational  awareness  enables 
collaboration  and  self -synchronization  and 
enhances  sustainability  and  speed  of  command. 

•  These  in  turn  dramatically  increase  mission 
effectiveness . 

These  tenets  facilitate  the  U.S.  military's 
implementation  of  NCW  across  four  domains:  physical, 
information,  cognitive,  and  social.  The  physical  domain 
encompasses  operations  such  as  strike,  protect,  and 
maneuver.  The  information  domain  includes  activities  of 
information  sharing,  creation,  and  manipulation.  The 
cognitive  domain  embodies  perceptions,  awareness,  beliefs, 
and  values.  Finally,  the  social  domain  recognizes 
interactions  between  and  among  organizational  elements. 
Miller  and  Shattuck  (2005)  refer  to  a  fifth  domain,  the 
ecological  domain.  The  ecological  domain  has  been  recently 
proposed  by  Lindh  (2004)  to  address  the  domain  of  context. 

As  Figure  1  illustrates,  the  domains  of  warfare 
conceptualized  in  U.S.  military  doctrine  overlap  with  one 
another.  An  analysis  of  each  domain  of  warfare  is  beyond 
the  scope  of  this  research.  However,  the  interaction  of 
the  social  and  cognitive  domains  (later  referred  to  as 
socio-cognitive)  comprise  information  exchange  and 
sensemaking  (Alberts  &  Hayes,  2003) ,  the  main  focus  of  this 
research . 
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Social  Domain 
Colaborative  Decisions 
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Cognitive  Advantage 
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Operations 


Shared 
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Deploy  Employ, 
and  Sustair 
Cycle 
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Precision 

Force  Advantage 

Force 

Position  Advantage 

Figure  1.  Domains  of  Warfare  Overlapping  (From:  Office  of 

Force  Transformation,  2005,  p.  21) 


The  socio-cognitive  domain  comprises  macrocognit ive 
behaviors  such  as  information  exchange  and  sensemaking 
(Figure  2,  NOW  Conceptual  Framework).  The  NOW  Conceptual 
Framework  is  the  result  of  a  2002  workshop  on  NOW  and 
networked  enabled  capabilities  (Evidenced  Based  Research, 
2003) .  This  framework  emphasizes  behaviors  which  are 
believed  to  be  critical  in  macrocognition.  Behaviors  such 
as  sensemaking  are  addressed  at  the  individual  and  team 
level  (Alberts,  Hayes,  &  Signori,  2001). 

At  the  team  level,  macrocognit ive  behaviors  include 
the  degree  to  which  (1)  information  is  shared  and  (2) 
shared  awareness  is  achieved.  Proponents  of  NOW  believe 
that  information  and  networking  "...form  the  center  of 
Network  Centric  Operations"  (Evidenced  Based  Research, 

2003,  p.  32) .  The  extent  to  which  teams  are  networked  (the 
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quality  of  networking)  along  with  their  ability  to  rapidly 
share  information  (the  quality  and  degree  of  information 
sharing)  are  said  to  be  critical  in  determining  overall 
effectiveness.  Sensemaking  involves  activities  that  allow 
individuals  to  "make  sense"  of  information  in  the  context 
of  experience,  use  this  information  to  make  inferences,  and 
organize  information  into  decisions  (Evidenced  Based 
Research,  2003).  Alberts  and  Hayes  (2003)  state  "...these 
variables  are  at  the  heart  of  the  collaborative  processes 
that  NCW  seeks  to  exploit"  (p.  99). 


Figure  1 1:  Top-Level  NCO  Conceptual  Framework,  Version  1.0 
Figure  2.  NCW  Conceptual  Framework  (From:  EBR,  2003,  p.  4) 

B .  PERSONALITY 

The  study  of  personality  in  characterizing  team 
effectiveness  or  understanding  military  adversaries  dates 
back  to  World  War  II.  During  this  time,  personalities  of 
enemy  nations  including  Germany,  the  Soviet  Union,  and 


10 


Japan  were  studied  by  anthropologists  at  the  direction  of 
the  U.S.  government  (Hofstede  &  McCrae,  2004) .  Trait 
psychology  emerged  as  a  contemporary  approach  to  the  study 
of  personality.  Trait  psychology  focuses  on  the  assessment 
of  individual  differences  in  enduring  dispositions  as 
predictors  of  behavior.  Dispositions  such  as  personality 
are  one  method  by  which  researchers  have  chosen  to  explain 
variability  in  team  performance. 

Central  to  the  study  of  trait  psychology  and  the 
characterization  of  personality  is  the  development  of  the 
Five-Factor  Model  (Digman,  1990).  The  Five-Factor  Model 
emerged  as  a  result  of  over  70  years  of  systematic  research 
beginning  with  efforts  to  organize  the  language  of 
personality  (John  et  al . ,  1988;  Digman,  1990).  Klages 
(1926)  suggested  that  a  careful  analysis  of  language  would 
assist  in  the  understanding  of  personality  which  lead  to 
Baumgarten' s  (1933)  examination  of  personality  terms 
commonly  found  in  the  German  language  (as  cited  in  Digman, 
1990)  . 

There  have  been  other  taxonomies  of  personality 
encompassing  as  few  as  three  factors  -  Psychot icism, 
Neuroticism,  and  Extrover sion/ Introversion  (Eysenck,  1960) 
and  as  many  as  16  factors  (i.e.,  Cattell's  Sixteen 
Personality  Factor  System;  Cattell,  1965).  As  cited  in 
Digman  (1990),  the  work  of  Fisk  (1949)  and  Tupes  and 
Christal  (1961)  helped  to  establish  the  superiority  of  a 
five-factor  approach.  Replicated  by  later  researchers  (for 
a  review  see  Digman,  1990),  the  Five-Factor  Model  has  been 
recognized  for  its  robustness.  Any  model  for  structuring 
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individual  differences  will  have  to  encompass  the  Big  Five 
dimensions  at  some  level  (Digman,  1990). 

The  Five-Factor  Model  consists  of  five  dimensions  of 
personality  (Digman,  1990),  often  referred  to  as  the  Big 
Five.  Personality  researchers  have  failed  to  reach  a 
consensus  on  the  names  representing  each  dimension  .  For 
this  thesis,  Costa  and  McCrae's  (1985)  five  personality 
constructs  are  used.  The  five  constructs  are  known  as 
extraversion,  agreeableness,  conscientiousness, 
neuroticism,  and  openness.  Within  each  dimension,  each 
construct  is  defined  by  at  least  six  specific  traits  or 
facets  which  are  addressed  in  further  detail  later  in  this 
section . 

Research  within  the  last  few  years  has  provided 
support  to  the  utility  of  using  personality  measures  as 
predictors  of  performance.  McHenry  et  al . ,  1990  (as  cited 
in  Neuman  &  Wright,  1999)  observed  that  personality  is  a 
predictor  of  job  performance  beyond  general  cognitive 
ability  and  job-specific  skills  for  Army  personnel  .  The 
following  sections  are  intended  to  provide  a  more  detailed 
explanation  of  each  of  the  Big  Five  and  its  implications 
for  assessing  individual  and  team  performance. 

The  Big  Five  factor  of  Conscientiousness  is  comprised 
of  competency,  dutifulness,  need  for  achievement,  self- 
discipline,  and  the  tendency  to  think  carefully  before 
acting  (Barrick  &  Mount,  1991) .  According  to  Barrick  and 
Mount  (1993),  conscientiousness  is  related  to  individual 
performance  regardless  of  task  requirements.  Later 
research  (Kichuck  &  Wiesner,  1997;  Neuman  &  Wright,  1999) 
suggested  a  relationship  between  conscientiousness  and  team 
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performance.  Teams,  whose  members  score  high  on  the  "need 
for  achievement",  one  facet  of  conscientiousness, 
outperform  teams  whose  members  scored  low  (see  French, 

1958;  Schneider  &  Delaney,  1972;  Zander  &  Forward,  1968) . 

Agreeableness  is  characterized  by  trust, 
straightforwardness,  altruism,  compliance,  modesty,  and  the 
ability  to  be  tender-minded  (Barrick  &  Mount,  1991) .  In 
their  review,  Neuman  and  Wright  (1999)  suggest  that 
agreeableness  is  a  predictor  of  job  performance  at  the 
individual  level  (see  Rose  et  al .  ,  1994;  Tett,  Jackson,  & 

Rothstein,  1991) .  At  this  level,  facets  of  agreeableness 
such  as  trust,  straightforwardness,  altruism,  and 
compliance  are  traits  that  are  desirable  for  the  social 
interactions  found  among  team  members  (Aronoff  &  Wilson, 
1985) .  Teams  characterized  by  agreeable  members  are 
expected  to  exhibit  interpersonal  skills  such  as  ability  to 
resolve  conflict  and  communicate  openly  in  a  manner  which 
promotes  information  exchange  and  sensemaking  (Aronoff  & 
Wilson,  1985;  Neuman  &  Wright,  1999;  Agarwal,  2003)  .  In 
their  analysis  of  the  human  dimension  of  NCW,  Warne  et  al . 
(2004)  cite  research  which  demonstrated  that  the  extent  to 
which  an  individual  trusts  another  significantly  impacts 
their  willingness  to  share  valuable  information  with  others 
(see  Fine  &  Holyfield,  1996). 

Extraversion  is  characterized  by  sociability, 
gregariousness,  assertiveness,  talkativeness,  and 
activeness  (Barrick  &  Mount,  1991)  .  Kickuk  and  Wiesner 
(1997)  cite  research  showing  that  extroversion  is 
positively  related  to  team  performance  (Haythorn,  1953; 
Ghiselli  &  Lodahl,  1958;  Smelser,  1961;  Altman  &  Haythorn, 
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1967;  Bouchard,  1969;  Shaw  &  Harkey,  1976;  Driskell  et  al . , 
1987;  Williams  &  Sternberg,  1988)  and  participation  with 
the  team  (Mann,  1959;  Watson,  1971) .  Extroverts  are  said 
to  help  facilitate  intra-team  communication  and  are 
generally  ready  to  share  information  (Bradley  &  Herbert, 
1997)  . 

Neuroticism  is  characterized  by  traits  such  as 
anxiety,  depression,  anger,  embarrassment,  emotionality, 
and  insecurity  (Barrick  &  Mount,  1991) .  Neuroticism  has 
also  been  thought  of  as  a  lack  of  emotional  stability  or 
adjustment  (Digman,  1990)  .  Neuroticism  has  not  been  found 
to  be  related  to  performance  (Kickuk  &  Wiesner,  1997) . 
However,  after  reviewing  studies  completed  by  Haythorn, 
(1953),  Man  (1959),  Shaw  (1971)  and  Thomas  et  al .  (1996), 

they  posit  that  Neuroticism  may  be  negatively  correlated 
with  team  performance. 

Traits  associated  with  Openness  include  imagination, 
culture,  curiosity,  originality,  broad-mindedness, 
intelligence,  and  having  an  artistic  ability  (Barrick  & 
Mount,  1991)  .  In  a  review  by  Kichuk  and  Wiesner  (1997)  , 
openness  to  experience  is  said  to  be  predictive  of  an 
individual's  training  proficiency  but  not  predictive  of 
his/her  performance.  The  relationship  between  openness  and 
overall  performance  is  inconclusive  (Kichuk  &  Wiesner, 

1997)  . 

C .  CULTURAL  ADAPTABILITY 

Culture  has  often  been  used  to  explain  human  behavior, 
but  "...there  is  no  single,  accepted  definition"  (Bird,  2001, 
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p.2) .  However,  there  are  characteristics  which  help 
conceptualize  culture.  Bird  (2001)  identifies  four  common 
characteristics : 

1.  Culture  includes  systems  of  values; 

2.  Culture  is  learned,  not  innate; 

3.  Culture  distinguishes  one  group  from  another;  and 

4.  Culture  influences  beliefs,  attitudes,  perception 
and  behavior  in  'somewhat'  uniform  and 
predictable  ways.  (pp.  2-3) 

In  the  context  of  NCW,  the  study  of  culture  may  be 
best  viewed  as  a  lens  (Trandis,  1994) .  The  ability  to 
examine  culture  through  a  lens  provides  the  flexibility  to 
study  culture  at  both  an  individual  and  team  level  .  This 
approach  also  allows  researchers  to  examine  the  dynamic 
relationships  that  may  exist  between  the  four  previously 
mentioned  characteristics  of  culture. 

Cultural  differences  affect  coordination  and 
communication,  which  are  factors  of  sense-making.  In  their 
review  of  culture  and  team  performance,  Powell,  Piccoli, 
and  Ives  (2004)  suggest  that  differences  in  culture  may 
lead  to  coordination  difficulties  (see  Johansson  et  al., 
1999;  Kayworth  &  Leidner,  2000;  Maznevski  &  Chdoba,  2001; 
Robey  et  al . ,  2000)  and  create  obstacles  to  effective 
communication  (see  Kayworth  &  Leidner,  2000;  Sarker  & 

Sahay,  2002;  van  Ryssen  &  Godar,  2000)  . 

Cultural  differences  within  teams  may  also  be 
magnified  in  network-centric  environments.  Networking 
implies  linking  individuals  together  to  work  in  teams. 

Thus,  networking,  from  a  human  sense,  ultimately  results  in 
teamwork.  Teamwork  connotes  individual  accountability, 
information  exchange,  and  a  general  sense  of  working 
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together  (Drucker,  1999) .  Consequently,  teamwork  is  viewed 
as  a  skill  and  its  absence  can  be  a  barrier  to  effective 
performance  (Warne  et  al .  ,  2004)  . 

Klein,  Pongonis,  and  Klein  (2000)  identify  five 
cultural  differences  that  can  disrupt  sensemaking, 
decision-making,  coordination,  and  communication  in 
military  team  work:  (1)  power  distance,  (2)  dialectical 
reasoning,  (3)  counterf actual  thinking,  (4)  risk  assessment 
and  uncertainty  management,  and  (5)  activity  orientation. 
This  research  focuses  on  only  one  of  these  five 
differences,  uncertainty  management. 

Uncertainty  is  "the  state  of  being  unsettled  or  in 
doubt"  (American  Heritage  Dictionary,  2000)  .  Uncertainty 
can  be  a  function  of  ambiguous  information,  perceived 
threat,  or  harmful  situation.  Greco  and  Roger  (2001) 
advocate  that  "...stress  under  these  conditions  varies  as  a 
function  of  an  individual's  efforts  of,  appraisal  of,  and 
coping  with,  the  event"  (p.  517) .  The  study  of  uncertainty 
and  human  behavior  has  become  known  variously  as  tolerance 
of  ambiguity,  uncertainty  avoidance,  anxiety,  and  risk 
avoidance .  These  terms  are  used  interchangeably  in  this 
thesis  . 

Uncertainty  has  been  conceptualized  and  studied  within 
a  variety  of  cultural  contexts .  At  the  national  culture 
level,  uncertainty  may  be  termed  uncertainty  avoidance. 
Cultures  classified  as  strong  in  uncertainty  avoidance  are 
active,  aggressive,  emotional,  compulsive,  security¬ 
seeking,  and  intolerant.  Other  cultures  that  can  be 
charact eri zed  as  low  in  uncertainty  avoidance  are 
contemplative,  less  aggressive,  unemotional,  relaxed. 
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accepting  personal  risks,  and  relatively  tolerant 
(Hofstede,  1991).  Uncertainty  avoidance  has  also  been 
"related  to  anxiety,  need  for  security  and  dependence  upon 
experts"  (Hofstede,  1980,  p.110). 

The  need  for  certainty  is  not  only  different  between 
cultures,  but  also  within  cultures.  As  a  trait,  the  need 
for  certainty  has  been  labeled  as  an  intolerance  of 
ambiguity.  Intolerance  of  ambiguity  has  been  defined  as 
"the  tendency  to  perceive  (i.e.,  interpret)  ambiguous 
situations  as  sources  of  threat"  (Budner,  1962,  p.  29)  . 
Ambiguous  situations  perceived  as  threats  include  new 
technology,  dynamic  and  complex  environments,  asymmetry  of 
the  battlefield,  and  dynamic  information  exchange  networks . 
Overall,  a  high  need  for  certainty  or  a  low  tolerance  for 
ambiguity,  imply  "a  preference  for  familiarity,  symmetry, 
definiteness,  and  regularity"  (Bar-Tal,  1994,  p.  45) . 

Stress  is  one  of  the  human  responses  to  uncertainty. 
Stress  can  be  initiated  by  a  distinct  event,  ambiguous  and 
incomplete  information,  or  a  perceived  threat  .  Stress  also 
has  an  effect  on  performance.  Performance  differences 
which  may  result  from  stress  include  premature  reactions, 
restricted  use  of  relevant  cues,  more  errors  on  cognitive 
tasks,  and  increased  use  of  schematic  or  stereotyped 
judgments  (see  Bar-Tal,  Raviv,  &  Spitzer,  1999) .  However, 
not  everyone's  reactions  to  stressors  are  the  same. 

A  12-month  team  performance  study  led  by  Sutton  and 
Pierce  (2003) ,  found  that  uncertainty  affected  situation 
assessment  (i.e.,  information  exchange),  coordination 
(i.e.,  response  sequencing),  and  general  support  behavior 
(i.e.,  activity  monitoring).  Their  study  assessed  the 
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degree  to  which  cultural  cognitive  dimensions  impact 
teamwork  in  a  multinational  headquarters.  Specifically, 
teams  comprised  of  individuals  scoring  high  on  need  for 
certainty  (uncertainty  avoidance)  engaged  in  behaviors  that 
were  different  from  team  members  with  a  low  need  for 
certainty.  Individuals  scoring  high  on  need  for  certainty 
produced  much  more  detailed  information  in  the  situation 
assessment  phase  of  planning  and  coordination.  These 
individuals  also  developed  well-defined  processes  for 
information  exchange  and  coordination. 

D.  OPTIMISM  AND  PESSIMISM 

Pessimism  is  defined  as  a  general  tendency  to  have 
negative  expectations.  "Optimism  is  a  disposition 
inclining  one  to  positive  expectations;  pessimism  inclines 
one  to  negative  expectations"  (Helton  et  al . ,  1999,  p. 

311) .  Optimists  and  pessimists  respond  to  stress  in 
different  ways.  Stress  narrows  attention  and  causes 
important  information  to  be  overlooked.  This  narrowing 
affects  the  problem  solving  process,  another  process 
related  to  macrocognition  (Hoffman,  Roesler,  &  Moon,  2004) . 
Central  to  the  activity  of  problem  solving  is  sensemaking 
(Russell  et  al .  ,  1993)  .  Seligman  (as  cited  in  Grasha, 

2000)  suggests  that  optimists  are  willing  to  challenge 
problems,  persist  until  a  solution  is  found,  and  involve 
others  in  resolving  issues  (Seligman,  1991) .  Optimists  are 
believed  to  participate  more  in  information  exchange. 

E.  MODEL  OF  DYNAMIC  SITUATED  COGNITION 

NCW  advocates  cite  the  need  for  a  process  model  to 
assess  NCW's  claims.  However,  such  a  process  model  has  not 
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been  available  until  recently.  Miller  and  Shattuck  (2005) 
proposed  filling  this  gap  using  their  Dynamic  Model  of 
Situated  Cognition  (DMSC)  (Figure  3) . 


The  Dynamic  Model  of  Situated  Cognition 


Figure  3.  The  Dynamic  Model  of  Situated  Cognition  (From: 

Miller  &  Shattuck,  2004) 

The  DMSC  emerged  as  an  attempt  to  illustrate  the 
relationships  between  technological  systems  and  human 
perception  and  cognition.  Since  its  creation,  the  DMSC  has 
been  used  to  describe  what  happens  in  operational 
environments  with  people  who  are  engaged  in  goal  directed 
behavior  (see  Miller  &  Shattuck,  2004,  2005).  It  has  also 
been  adapted  by  Miller  and  Shattuck  (2005)  as  a  process 
model  for  NCW  (Figure  4) .  The  DMSC  can  be  used  to  examine 
macrocognit ive  processes  and  functions  of  information 
exchange  and  sensemaking  which  reside  in  the  socio- 
cognitive  domain.  Situated  cognition  describes  the  fact 
that  macrocognit ive  functions  are  generally  performed  in 
collaboration  by  a  team  working  in  a  natural  setting. 
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Accordingly,  the  left  side  of  the  model  receives  the 
greatest  attention  in  this  research. 


The  Dynamic  Model  of  Situated  Cognition 
with  NCOW  Domains  of  Warfare 


Physical  and  Information  Domains  Cognitive  Domain 


Figure  4 .  The  DMSC  adapted  to  the  NCOW  Domains  of  Warfare 

(From:  Miller  &  Shattuck,  2005) 


The  DMSC  consists  of  a  series  of  lens  (A,  B,  and  C, 
see  Figure  3) .  Lens  A,  the  lens  between  Ovals  3  and  4, 
directs  attention  to  selected  incoming  stimuli.  These 
stimuli  are,  in  most  cases  either  visual  or  auditory. 
Between  Ovals  4  and  5  is  Lens  B.  Lens  B  is  believed  to 
influence  how  data  are  organized  into  information.  The 
lens  between  Ovals  5  and  6,  Lens  C  is  believed  to  guide  the 
process  of  extrapolating  current  information  (Shattuck  & 
Miller,  2005)  . 

Miller  and  Shattuck  (2005)  state  that  there  are  at 
least  six  classes  of  information  embedded  in  the  lenses 
that  influence  macrocognit ive  processes.  The  six  classes 
of  information  are:  1)  individual  states  and  traits,  2) 
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social  factors,  3)  local  context,  4)  plan,  5)  guidelines, 
and  6)  experience.  Two  are  applicable  in  the  context  of 
this  research:  individual  states  and  traits,  and  social 
factors.  Individual  states  and  traits  represent  relatively 
enduring  (e.g.,  personality)  and  transient  (e.g.,  fatigue) 
characteristics  of  an  individual .  For  a  more  detailed 
discussion  of  human  trait  and  state  measurements,  see 
Miller  et  al .  (2003) .  Social  factors  include  issues 

ranging  from  team  dynamics  (e.g.,  homogeneity)  to  cultural 
differences.  Together  these  two  factors  may  significantly 
influence  what  is  perceived  by  team  members  (Oval  4) . 

The  ovals  to  the  right  of  lens  A  (Oval  4,  5,  and  6) 
represent  perception,  comprehension,  and  predictions  - 
loosely  referred  to  as  macrocognit ive  processes. 

Perception  (a  process  occurring  in  Oval  4)  is  achieved 
through  active  and  passive  processes  such  as  information 
exchange.  Active  processes  refer  to  data  requested  or 
"pulled"  by  team  members.  Passive  processes  refer  to  data 
provided  or  "pushed"  to  team  members.  Comprehension  is 
represented  in  Oval  5.  Comprehension  is  a  cognitive  process 
described  by  terms  such  as  fusion,  integration,  analysis, 
explanation,  interpretation,  and  pattern  recognition 
(Endsley,  1995) .  Comprehension  is  loosely  referred  to  as 
sensemaking.  Dervin' s  theory  of  sensemaking,  (as  cited  in 
Salvolainen,  1993)  is  defined  as:  "...  behavior,  both 
internal  (i.e.,  cognitive)  and  external  (i.e.  procedural), 
which  allows  the  individual  to  construct  and  design  their 
movement  through  time-space"  (p.16)  .  Finally,  Oval  6 
represents  projections  of  individual  team  members.  These 
projections  are  based  on  what  have  been  comprehended  (Oval 
5),  and  the  affect  of  an  individual's  lens  (Lens  C) .  These 
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same  lens  components  (i.e.,  individual  states  and  traits 
and  social  factors)  are  believed  to  influence 
macrocognit ive  processes  such  as  information  exchange  and 
sense-making . 

NCW,  by  its  very  nature,  involves  multiple  actors, 
both  human  and  machine.  Miller  and  Shattuck  (2005)  state 
"As  we  move  from  an  individual  to  the  network  of 
individuals  that  characterize  NCW"  the  technological  (or 
physical  and  information  domains)  remain  the  same.  Figure 
5  below  illustrates  how  ground  truth  (Oval  1)  and  data 
detected  by  sensor  systems  (Oval  2)  will  remain  the  same 
for  each  individual.  However,  Oval  3  through  Oval  6  will 
differ  for  each  of  the  three  individual  team  members.  As 
an  illustration,  the  lines  from  individuals  A  and  C  to 
individual  B  represent  the  way  individuals  A  and  C  convey 
information  to  individual  B.  The  same  descriptions  of 
lenses  and  ovals  are  applicable  in  this  illustration. 


Figure  5.  DMSC  Applied  to  Teams  (From:  Miller  &  Shattuck, 
2005,  Published  in  Conference  Proceedings 
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Situated  cognition  is  a  dynamic,  ongoing  process 
(Clancey,  1997) .  Figure  6  further  represents  this  ongoing 
process  by  its  emphasis  on  feedback  in  situated  cognition. 
The  feedback  loops  shown  in  Figure  6  provide  insight  into 
macrocognit ive  processes  of  networked  individuals.  The 
feedback  loops  flow  from  Oval  5  (comprehension)  to  Ovals  1, 
2,  3,  4  (environmental,  sensors,  C2  workstation,  and 
perception) .  Feedback  loops  from  Oval  6  (projection)  also 
flow  to  Ovals  1,  2,  3,  4,  and  5.  Additional  feedback  loops 
exists  in  the  model  as  well .  These  feedback  loops  extend 
from  Oval  5  (comprehension)  to  the  lenses  and  from  Oval  6 
(projection)  to  the  lenses.  Miller  and  Shattuck  (2005) 
have  postulated  that  the  lenses  are  dynamic  and  constantly 
change . 


A  Dynamic  Model  of  Situated  Cognition 


Figure  6.  DMSC  Feedback  Loops,  (From:  Miller  &  Shattuck, 

2004,  p . 5 ) 
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The  factors  influencing  effective  macrocognit ive  processes 
must  be  investigated  insofar  as  they  underpin  mission 
effectiveness  (Warne  et  al .  ,  2005)  . 


2  4 


III.  DESIGN  AND  IMPLEMENTATION  OF  THE  TEAM 
PERFORMANCE  LABORATORY 


A .  BACKGROUND 

Much  of  the  time  involved  in  this  thesis  effort  was 
spent  designing,  constructing,  and  configuring  the  Team 
Performance  Laboratory.  The  Team  Performance  Laboratory 
uses  computer  software  to  assess  and  evaluate  human  factors 
considerations  in  team  performance.  The  Human  Systems 
Integration  Laboratory  (HSIL)  was  reconfigured  to 
accommodate  two  semi-private  four  person  laboratories. 
Additionally,  four  mobile  desktop  stations  were  configured 
for  easy  set-up  and  take-down  given  limited  laboratory 
space.  A  two  person  observer  station  was  set  up  to 
network,  control,  observe,  and  collect  data  on  the  eight 
Pentium  4  desktop  computers  used  by  team  members  in  the 
C3Fire  simulation. 

B .  SOFTWARE 

Testing  the  claims  of  NCW  using  a  computer  generated 
synthetic  environment  required  reviewing  numerous  game- 
based  software  packages.  These  included  Mission  to  Mars, 
AWACS-AEDGE,  Never  Winter  Nights  -  SABRE,  and  C3Fire. 

Mission  to  Mars  is  a  computer-generated  interactive 
simulation  currently  being  used  by  researchers  to  evaluate 
distributed  interactive  communication  between  dispersed 
elements.  The  simulations  are  developed  around  resource 
and  time  management  themes  within  the  context  of  geologic 
exploration  expeditions  on  the  surface  of  Mars.  Groups  of 
three-person  crews  are  expected  to  operate  a  simulated 
exploration  vehicle,  an  "Orbiter",  "Lander",  or  "Rover", 
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via  an  individual  workstation.  The  task  is  designed  to 
last  up  to  four  hours.  Mission  to  Mars  automatically 
records  both  crew  performance  and  psychosocial 
communication  interactions  (Hienz  et  al . ,  in  press). 

The  Airborne  Warning  and  Control  System  -  Agent 
Enabled  Decision  Guide  Environment  (AWACS-AEDGE ) ,  developed 
using  21st  Century  Systems  Inc.'s  AEDGE™  infrastructure  is  a 
distributed,  real-time  team  decision  support  environment. 

It  is  comprised  of  simulators,  intelligent  agents  and  user 
interfaces  (Barnes,  Petrov,  &  Elliott,  2002)  .  The  AWACS- 
AEDGE  was  developed  to  represent  core  characteristics  of 
the  Airborne  Warning  and  Control  Systems  (AWACS)  Weapons 
Director  (WD)  team.  Core  characteristics  of  the  AWACS-WD 
team  include  providing  airborne  surveillance,  control,  and 
communications  functions  for  tactical  and  air  defense 
forces.  The  AWACS-AEDGE  was  developed  as  an  agent-based  C2 
team  decision  support  platform  for  research  and  training 
(Petrov  et  al . ,  2002) . 

SABRE  (Situation  Authorable  Behavior  Research 
Environment)  is  a  joint  Defense  Modeling  and  Simulation 
Office  and  Air  Force  Research  Laboratory  project. 

Developed  by  BBN  Technologies,  SABRE  is  a  tool  intended  for 
team  behavior  research.  SABRE  is  marketed  as  an  aid  for 
investigating  general  aspects  of  teamwork  such  as  group 
decision-making,  resource  management,  and  information 
sharing.  In  addition,  context -specif ic  behaviors  such  as 
negotiating  and  accommodating  mission-irrelevant  requests 
for  assistance  are  explored.  This  test-bed  has  been 
developed  primarily  to  study  the  effects  of  personality  and 
culture  on  behavior  and  performance  in  a  cooperative  team 
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mission  (Leung,  Diller,  &  Ferguson,  2005).  Training  time 
prior  to  data  collection  is  approximately  2.5  hours  (Warren 
et  al . ,  2005)  . 

The  C3Fire  microworld  is  a  command,  control,  and 
communicat ions  firefighting  simulation.  C3Fire  has  been 
used  for  training  and  experimentation  of  team  decision¬ 
making  and  team  situational  awareness  (Granlund  et  al . , 
2001;  Granlund,  2002)  .  C3Fire  generates  a  task  environment 
in  which  a  team  of  four  people  cooperate  to  extinguish  a 
fire  (see  Figure  7) .  The  user  interface  consists  of 
several  basic  elements:  a  geographic  information  system 
(GIS) ,  a  diary,  and  an  e-mail  system.  In  the  center  of  the 
user  interface  is  a  map  consisting  of  a  40  x  40  matrix  of 
cells;  a  map  legend,  clock,  e-mail  tool,  and  a  truck  status 
panel.  Using  these  features,  players  of  C3Fire  control 
three  types  of  trucks  (firefighting,  fuel,  and  water)  and 
are  responsible  for  working  together  via  e-mail  to 
extinguish  the  fire.  Players  need  to  maintain  a  picture  of 
fuel  and  water  states  during  the  game .  The  game  records  a 
variety  of  performance  data  in  the  form  of  logs.  Training 
time  is  approximately  2  minutes. 
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Figure  7.  An  example  of  C3Fire  Microworld  Setting,  (From 
C3Fire  Instructions  Granlund,  2005,  p.  1) 

After  reviewing  the  concepts  of  NCW  and  given  the 
short  amount  of  training  time,  C3Fire  was  chosen.  C3Fire 
was  also  selected  because  it  had  been  previously  used  to 
test  concepts  of  NCW  (Johansson  et  al.,  2003)  . 


C .  INSTALLATION 

C3Fire  was  installed  on  ten  desktop  Pentium  4 
machines.  In  addition  to  the  software,  basic  requirements 
for  the  team  performance  lab  included: 

•  Two  adjacent  areas  (Team  Performance  Laboratory) 
measuring  at  least  14'  X  14' .  Each  area  should 
be  fully  isolated  from  other  areas  in  the  HSIL  to 
protect  against  background  noise.  Each  area  may 
require  as  many  as  twelve  surge-protected 
electrical  outlets. 

•  Three  10/100  Ethernet  ports  capable  of  networking 
four  computers  at  once. 

•  Ten  Pentium  4  or  3  CPU  with  near  3.20GHz,  1.00  GB 

of  RAM. 

•  Ten  standard  "qwerty"  keyboards  and  two  button 
mouse.  Ten  monitors  of  the  same  size  are  needed. 
Monitors  should  be  no  smaller  than  a  17-inch 
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Diagonal  and  16-inch  V.I.S.,  CRT,  1024x768 
Pixels.  Ideally,  19"  or  20"  Flat  Panel  LCD 
Monitors  are  required  for  resolution  and 
interface  visibility. 

All  machines  need  to  be  configured  by  the 
Information  Technology  Assistance  Center  (ITACS) 
to  receive  a  constant  IP  Address  and  connect  to 
the  NPS  Intranet. 
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IV.  PILOT  STUDY 


Identifying  and  measuring  human  performance  both  on  an 
individual  and  team  level  is  challenging.  Team  performance 
variables  must  be  operationally  defined,  using  the 
constructs  identified  for  specific  operations  or  procedures 
and  the  resultant  measurements  selected  for  this  study. 

Key  variables  selected  for  this  study  and  their  operational 
definitions  are  provided  below.  A  major  benefit  of  C3Fire 
and  a  deciding  factor  for  its  use  in  this  study  is  its 
capability  to  automatically  collect  several  kinds  of 
performance  data.  The  primary  variable  of  interest  in  this 
thesis,  information  exchange  (or  information  currency),  is 
an  indicator  of  overall  NCW  performance  which  has  been 
previously  identified  in  the  literature.  In  the  context  of 
NCW,  information  exchange  (information  currency)  is 
consistent  with  objective  measures  of  information  sharing 
and  sensemaking  at  the  team  and  individual  levels.  For 
more  information  on  NCW  variables  in  addition  to  the  ones 
which  have  been  identified  below,  see  Appendix  B.  For 
C3Fire  capabilities,  refer  to  www .C3Fire.org. 

A.  VARAIABLES 

1 .  Dependent  Variables 

a.  Information  exchange  (information  currency): 
communicat ion  lag  time  (in  seconds)  between 
sending  and  the  receiver  opening  a  message. 

b.  Overall  C3Fire  Performance  -  the  number  of 
cells  in  the  40  x  40  matrix  of  cells  lost 
due  to  ineffective  firefighting  in  a  17 
minute  session. 
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2 .  Independent  Variables 

a.  Personality  scores  on  each  of  five 
dimensions  -  measured  using  the  NEO-FFI. 

b.  Uncertainty  Management  scores  on  each  of 
three  scales:  (1)  Uncertainty  Response 
Scale,  (2)  Need  for  Cognitive  Structure,  and 
(3)  Ability  to  Achieve  Cognitive  Structure 

c.  Optimism-Pessimism  -  total  score  from  the 
OP  I . 

d.  Conflict  Avoidance  -  total  score  from  the 
Conflict  Avoidance  questionnaire. 

B .  PARTICIPANTS 

Much  of  the  previous  research  on  teams  has  used 
undergraduate  students  from  civilian  institutions,  thereby 
threatening  generalizability  of  the  studies  for  U.S. 
military  forces.  Therefore,  the  current  study  specifically 
sought  participants  from  a  population  of  military  officers 
expected  to  perform  in  network-centric  operations. 

Thirty-two  NPS  students  and  faculty  members  (average 
age  =  34.1,  standard  deviation  =  6.12)  were  assigned  to 
eight  teams  who  participated  in  the  first  phase  of  the 
exercise.  Sixteen  additional  participants  (both  NPS 
students  and  faculty)  participated  but  data  from  these 
latter  participants  are  not  included  in  this  pilot  study 
due  to  incomplete  survey  responses. 

The  pilot  study  focused  on  the  performance  of  eight 
teams.  Of  the  eight  teams,  five  were  all  male.  Two  teams 
were  of  mixed  gender  and  one  team  was  all  female. 

Individuals  in  each  team  were  known  to  each  other. 
Naval  Postgraduate  School  students  enter  as  cohorts  taking 
the  same  classes  for  18  to  28  months,  making  it  virtually 
impossible  to  avoid  teams  consisting  of  individuals  who  had 
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no  prior  experiences  of  working  together.  Consequently, 
each  team  was  self-selected  and  entered  the  laboratory  as  a 
unit . 


The  32  participants  represented  most  branches  of 
service  including  civilians  from  the  DoD .  See  Table  1  for 
more  detailed  demographic  data. 


Navv 

Armv 

Marines 

Air  Force 

Coast  Guard 

DoD  Civilian 

Total  i 

16 

6 

1 

3 

0 

6 

32 

Table  1 .  Military  Service  Representation 


C .  INSTRUMENTS 

A  variety  of  standardized  surveys  were  used  for 
collecting  individual  characteristics  and  trait  data.  The 
surveys  selected  for  this  research  were  a  mix  of  open-  and 
closed-ended  questions  that  were  3-5  pages  in  length.  For 
consistency  and  ease  of  scoring,  many  of  the  surveys  were 
scored  using  a  five-point  Likert  scale  (1  =  Strongly  Agree 
to  5  =  Strongly  Disagree) . 

The  surveys  listed  below  were  selected  because  of 
their  perceived  ability  to  be  related  to  team  performance. 
The  number  of  surveys  selected  was  indicative  of  the 
absence  of  well  defined  individual  characteristics  and 
traits  that  are  related  to  team  performance  literature  (See 
Chapter  II:  Literature  Review) .  Refer  to  the  references 
listed  for  additional  information  on  and  a  copy  of  each 
survey . 

1 .  Demographic  Questionnaire 

The  demographics  questionnaire  is  a  15-item 
questionnaire  with  both  open-ended  and  f orced-choiced 
questions  which  assesses  the  participants''  (1)  personal. 
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academic,  and  work-related  background,  (2)  international 
experiences,  (3)  experience  in  the  military  and  teamwork, 
and  (4)  experience  with  computers,  especially  chat 
programs.  The  15-item  questionnaire  was  adapted  from 
ongoing  research  on  Bridging  Cultural  Barriers  to 
Collaborative  Decision  Making  in  Onsite  Operations 
Coordination  Centers  (K.  Smith,  personal  communication, 
August  22,  2005,  11:32PM) . 

2.  NEO  Five-Factor  Inventory  (NEO-FFI) 

Team  member  personality  was  measured  using  the  NEO- 
FFI,  a  60-item  questionnaire.  It  is  the  shortened  version 
of  the  Revised  NEO-Personality  Inventory  (NEO-PI-R;  Costa 
and  McCrae,  1992) .  The  NEO-FFI  provides  an  accurate  and 
concise  measure  of  the  "Big  Five"  domains  of  adult 
personality:  extroversion,  agreeableness, 

conscientiousness,  emotional  stability,  and  intellect  (or 
openness)  and  their  facets  (Briggs,  1989) . 

The  NEO-FFI  has  adequate  internal  consistency, 
construct,  and  discriminative  validity  across  diverse 
samples  (Ball  et .  al . ,  2001;  Costa  &  McCrae,  1992).  The 
NEO-FFI  Form  S  correlates  with  the  NEO-PI-R  domain  scales 
at  .77-. 92  and  has  a  internal  consistency  values  ranging 
from  .68-. 86  (PAR,  Inc.,  Retrieved  2005).  Additionally, 
the  NEO-FFI  was  chosen  over  other  popular  indices  of 
personality  because  of  its  relational  value  to  national 
cultural  constructs  previously  mentioned  in  the  literature 
(Hofstede  &  McCrae,  2004)  and  its  10-15  minute 
administration  time. 
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3 .  Uncertainty  Response  Scale 

The  Uncertainty  Response  Scale  (URS;  Greco  &  Roger, 
2001)  is  a  48-item  scale  which  predicts  individual 
differences  in  coping  with  uncertainty.  The  URS  is 
comprised  of  three  factors.  Emotional  Uncertainty  (EU) , 
Desire  for  Change  (DFC) ,  and  Cognitive  Uncertainty  (CU) . 

EU  is  the  desire  to  which  an  individual  responds  to 
uncertainty  with  anxiety  and  sadness.  DFC  is  the  degree  to 
which  an  individual  enjoys  novelty,  uncertainty,  and 
change.  CU  is  the  degree  to  which  an  individual  prefers 
order,  planning,  and  structure  in  an  uncertain  environment. 
Participants  rate  the  degree  to  which  a  statement  is  true 
for  themselves  using  a  5-point  scale  with  endpoints  being  1 
=  Never  and  5  =  Always.  Scores  for  subscales  are 
determined  by  totaling  the  point  value  of  statements 
associated  with  each  subscale.  Higher  scores  indicate  a 
greater  tendency  toward  maladaptive  responses  to 
uncertainty  (EU) ,  greater  enjoyment  of  the  unknown  (DFC) , 
and  greater  preference  for  control  under  uncertain 
conditions  (CU) .  The  URS  has  an  internal  consistency  of 
.89,  .90,  and  .85  for  subscales  EU,  DFC,  and  CU 

respectively,  and  test-retest  reliability  estimates  of  .79, 

. 86,  and  . 80 . 

4 .  The  Need  for  Cognitive  Structure  Scale 

The  Need  for  Cognitive  Structure  Scale  (NCS;  Bar-Tal, 
1994)  is  a  20-item  scale  that  assesses  the  extent  to  which 
an  individual  prefers  using  cognitive  structuring  to 
achieve  certainty.  Cognitive  structuring  (or 
categorization)  helps  create  certainty  by  filtering  out 
inconsistent  or  irrelevant  information.  Participants  rate 
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the  degree  to  which  they  disagree  or  agree  with  statements 
using  a  5-point  Likert  scale  with  endpoints  of  1  =  Strongly 
Disagree  and  5  =  Strongly  Agree.  Responses  are  totaled  to 
create  an  "overall  need  for  cognitive  structure"  score. 
Higher  scores  indicate  a  greater  need  for  cognitive 
structure.  The  NCS  has  an  internal  consistency  of  .82  and 
test-retest  reliability  of  .85  (Bar-Tal,  1993,  1994). 

5 .  The  Ability  to  Achieve  Cognitive  Structure  Scale 

There  are  individual  differences  in  the  ability  to 
effectively  organize  information  to  fit  existing  knowledge 
structures  or  to  process  information  that  is  inconsistent 
with  existing  structures.  The  Ability  to  Achieve  Cognitive 
Structure  Scale  (AACS;  Bar-Tal,  1994)  is  a  24-item  scale 
that  assesses  this  trait. 

Participants  rate  the  degree  to  which  they  disagree  or 
agree  with  statements  using  a  5-point  Likert  scale  with 
endpoints  of  1  =  Strongly  Disagree  and  5  =  Strongly  Agree. 
Responses  are  totaled  to  create  an  overall  ability  to 
achieve  cognitive  structure  score.  Higher  scores  indicate 
a  greater  ability  to  apply  information  processes  that  are 
consistent  with  an  individual's  level  of  NCS.  The  AACS  has 
an  internal  consistency  of  .67  and  a  test-retest 
reliability  of  .86  (Bar-Tal,  1993,  1994). 

6.  Optimism-Pessimism  Instrument 

The  Optimism-Pessimism  Instrument  (OPI)  is  a  56-item 
questionnaire.  Optimism  is  a  disposition  representing  a 
bias  toward  positive  aspects  of  life.  Pessimism  is  a 
disposition  representing  a  bias  toward  negative  aspects  of 
life.  In  the  OPI,  18  items  indicate  optimism  (0),  18  items 
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indicate  pessimism  (P),  and  20  items  are  filler.  The  scale 
was  developed  to  measure  the  degree  to  which  an  individual 
is  either  an  optimist  or  pessimist  in  their  expectations 
(Dember  et  al .  ,  1989) . 

Respondents  are  asked  to  rate  their  agreement  with  the 
items  using  a  four-point  Likert-type  scale  with  endpoints 
of  1  =  Strongly  Agree  and  4  =  Strongly  Disagree.  Unlike 
other  measures  of  optimism-pessimism  such  as  the 
Attribut ional  Style  Questionnaire  (ASQ;  Peterson  et  al .  , 
1982)  and  the  Life  Orientation  Test  (LOT;  Scheier  &  Carver, 
1985),  OPI  measures  pessimistic  and  optimistic  tendencies 
separately . 

The  OPI  has  an  internal  consistency  of  .84  for 
optimism  and  .87  for  pessimism.  Test-retest  reliability 
are  .75  and  .84  for  optimism  and  pessimism  respectively. 

7 .  Conflict  Avoidance  (CA) 

The  Conflict  Avoidance  (CA)  scale  is  a  23-item  self- 
report  measure  that  assesses  a  person's  reaction  to 
conflict.  High  scores  on  this  survey  indicate  a  tendency 
for  wanting  to  avoid  conflicts.  The  scale  is  comprised  of 
items  from  the  NEO-PI-R,  the  Intercultural  Adaptation 
Potential  Scale  (ICAPS;  LeRoux,  J.  &  Matsumoto,  D.,  2000), 
the  ROAD,  and  an  additional  Conflict  Avoidance  scale  taken 
from  (Tjosvold,  1985;  Barker,  Tjosvold,  &  Andrews,  1988). 

D .  PROCEDURE 

Data  collection  was  completed  in  two  phases.  In  the 
first  phase  of  data  collection,  eight  teams  of  four  NPS 
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faculty  and  students  were  asked  to  participate  in  a  single 
90  minute  exercise.  Each  laboratory  exercise  took  place  at 
the  HSIL. 

At  the  beginning  of  each  exercise,  participants  were 
asked  to  register  and  pick-up  a  manila  folder  containing 
their  informed  consent  form  and  surveys.  After  each 
participant  completed  their  informed  consent  form,  they 
began  completing  three  of  the  seven  surveys.  After 
completing  the  surveys,  participants  were  asked  to  read  the 
C3Fire  instructions.  Once  everyone  completed  their  surveys 
and  had  a  chance  to  read  over  the  C3Fire  instructions,  each 
of  the  four  team  members  moved  into  position  on  a  four 
person  station.  Participants  played  C3Fire  as  a  team 
training  exercise  for  three  minutes.  Afterwards, 
instructions  were  reviewed  and  participants  were  given  the 
opportunity  to  familiarize  themselves  with  C3Fire.  During 
this  3  minute  exercise,  questions  were  encouraged  and 
answered  aloud  to  ensure  participants  were  familiar  with 
the  game . 

When  the  training  period  ended,  participants  were 
given  another  opportunity  to  ask  questions.  Then,  the 
first  17  minute  data  collection  session  began.  At  the  end 
of  the  first  seventeen  minute  session,  participants  were 
instructed  to  finish  the  remaining  four  surveys.  After 
each  participant  answered  all  surveys,  the  final  17  minute 
C3Fire  session  started. 

The  first  phase  of  this  pilot  study  consisted  of  self- 
organized  teams.  Results  from  these  teams  were  used  to 
characterize  teams  based  on  individual  characteristics  and 
traits  and  to  relate  performance  to  team  characteristics. 
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The  second  phase  of  this  study  used  pre-selected  teams 
designed  by  the  researcher.  These  participants  were  asked 
to  return  for  a  second  17-minute  play  of  C3Fire.  The 
original  self-organized  teams  were  replaced  by  teams 
designed  on  the  basis  of  their  individual  characteristics. 
Four  teams  were  constructed  on  the  basis  of  agreeableness 
(High  vs.  Low)  and  cognitive  uncertainty  (High  vs.  Low) . 

The  personality  trait  of  agreeableness  was  measured  using 
the  NEO-FFI .  Individual  responses  to  CU  were  collected 
using  the  Uncertainty  Response  Scale  (URS) .  The  NEO-FFI 
classifies  individuals  into  one  of  five  categories  (Very 
Low,  Low,  Average,  High,  and  Very  High)  for  each 
personality  trait,  providing  a  mechanism  for  assessing  the 
degree  to  which  a  personality  trait  is  present.  The  URS 
scores  an  individual's  degree  to  which  they  respond  to 
uncertainty.  Using  all  32  responses  on  the  URS,  summary 
statistics  were  calculated.  Upper  and  lower  bounds  were 
derived  for  responses  to  CU. 
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V.  RESULTS  OF  PILOT  STUDY 


A.  SUMMARY  STATISTICS  OF  PERSONALITY  TRAITS 

The  first  step  of  this  analysis  summarizes  individual 
characteristics  and  traits  from  the  sample  of  NFS  students 
and  faculty.  The  sample  consisted  of  thirty-two 
participants.  Responses  to  the  NEO-FFI,  the  personality 
measures  used  in  the  study,  were  organized  into  five- 
factors  with  five  levels  for  each  factor.  Participants 
were  classified  as  either  "Very  High"  to  "Very  Low"  in  each 
of  five  personality  factors  of  extroversion,  openness, 
agreeableness,  neuroticism,  and  conscientiousness. 
Distributions  of  each  personality  characteristic  are  given 
in  Table  2  .  Across  the  dimensions  of  personality,  the 
classification  "Average"  was  the  most  predominant  with 
thirty-one  percent  of  all  responses. 


Extroversion 

Openness 

Openness 

Conscientiousness 

2 

2 

4 

6 

4 

Low 

2 

4 

7 

13 

8 

14 

11 

1 1 

8 

6 

HEIM 

10 

11 

8 

4 

11 

mem 

4 

4 

2 

1 

3 

Total  32  32  32  32  32 

Table  2.  Classification  of  Personality  Type  and  Strength 

of  Classification 


Plots  similar  to  Figure  8,  are  computed  for  each 
individual  characteristic  and  trait  measure  (See  Appendix 
E-F) .  These  plots  check  for  abnormalities  in  responses  to 
each  survey.  As  Figure  8  demonstrates,  responses  to  each 
survey  were  generally  varied  and  none  of  the  responses 
stand  out  as  anomalous. 
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Response  to  Uncertainty 


Emotion  Uncertainty 
Desire  for  Change 
Cognitive  Uncertainty 


1  3  5  7  9  11  13  15  17  19  21  23  25  27  29  31 

Participants 

Figure  8.  Scores  on  the  Uncertainty  Response  Scale  (URS)  by 

Participant 

B.  SUMMARY  STATISTICS  FOR  PERFORMANCE 

Next  we  examine  team  performance.  Figure  9  plots  the 
lag  time  averaged  over  the  number  of  messages  sent  in  each 
exercise  by  team  for  both  trial  1  and  trial  2.  It 
illustrates  overall  performances  of  seven  of  the  eight 
teams.  One  team's  performance  is  omitted  because  its 
members  had  more  experience  with  actual  play  of  the  game. 
Time  lag  is  a  performance  measure  of  information  exchange 
and  represents  currency  of  information.  Smaller  time  lag 
represents  more  current  and  relevant  information.  This 
variable  is  believed  to  be  critical  to  team  performance  in 
dynamic  environments .  Having  more  current  information  may 
improve  mission  effectiveness  in  network  environments. 

Thus,  currency  of  information  is  a  performance  measure  for 
mission  effectiveness  in  NCW  (Effective  Based  Research, 

2003)  and  the  performance  variable  of  choice  in  this  study. 
In  both  trials.  Team  4  had  the  smallest  average  time  lag 
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and  thus  outperformed  the  other  teams  with  respect  to 
information  exchange  (Figure  9) . 


Average  Lag  Time  by  Team 


Trial  1 
Trial  2 


Teams 


Figure  9.  Information  Exchange  Between  by  Teams 

Overall  Performance: 

Number  of  Cells  Lost  Due  to  a  Lack  of  FireFighting 
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Figure  10.  Overall  Team  Performance,  Lost  Cells 

As  Figure  10  illustrates.  Team  3  outperformed  all 
other  teams  in  overall  performance.  Overall  performance  is 
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defined  as  the  number  of  cells  lost  due  to  ineffective 
firefighting . 

The  analyses  and  graphs  provided  in  the  remaining 
chapters  of  this  thesis  are  from  trial  1  data  only.  No 
comparisons  were  made  between  trials  due  to  learning 
effects  and  strategy  changes  within  teams.  As  illustrated 
in  Figures  9  and  10,  Team  3's  average  time  lag  increased 
between  trials,  but  overall  performance  remained  relatively 
superior  among  teams  for  both  trials  1  and  2.  Further 
analysis  from  communication  logs  showed  that  one  member  of 
Team  3  failed  to  check  messages  from  other  teammates  until 
the  final  minutes  of  play  in  trial  2.  Thus,  Team  3's 
average  time  lag  increased  and  forced  the  remaining  three 
teammates  to  reorganize  quickly  to  cover  for  the  lack  of 
communication.  With  Team  3's  prior  experience  in  trial  1, 
they  were  able  to  remain  effective.  Because  the  experiment 
was  not  designed  to  account  for  the  effects  of  learning, 
summary  data  from  trial  2  beyond  what  has  previously  been 
presented  is  not  included. 

Individual  performance  drives  team  performance. 
Individual  performance  within  teams  may  provide  insights 
into  variance  in  team  performance.  In  each  exercise, 
average  time  lag  was  recorded  for  each  participant  within 
each  team.  In  C3Fire,  a  message  is  hidden  and  therefore 
cannot  be  read  until  a  player  clicks  the  "next  button". 

The  assumption  here  is  that  when  a  player  "clicks  the  "next 
button",  the  message  is  read.  C3Fire  time  stamps  these 
events  and  stores  who  sends  and  reads  their  messages.  The 
time  between  sending  and  receiving  messages  is  termed  time 
lag.  In  a  seventeen-minute  session,  a  player  can  send  and 
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receive  a  number  of  messages.  It  is  the  responsibility  of 
the  player  to  ensure  they  remain  current  on  information  as 
it  could  affect  individual  and  team  performance,  requiring 
that  players  check  their  mail  regularly.  For  purposes  of 
this  study,  information  is  not  exchanged  until  the  process 
of  sending  and  reading  a  message  is  completed. 

Figure  11  shows  individual  performance  differences  in 
time  lag.  The  box  plot  shows  that  team  members  in  Team  4 
are  less  variable  in  managing  information  exchanges  while 
members  in  Teams  2  and  5  are  more  variable.  Following  the 
assumption  that  individual  performance  is  related  to  team 
performance.  Team  2  was  not  expected  to  have  the  best 
performance;  it  did  not. 


Green,  Team  1 
Light  Blue,  Team  2 
Yellow,  Team  3 
Purple,  Team  4 
Orange,  Team  5 
Red,  Team  6 
Blue,  Team  7 


Figure  11.  Box  plots  of  Message  Time  Lags  by  Participant 
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Time  lag  was  also  measured  at  the  team  level.  Time 
lag  measured  at  the  team  level  is  postulated  to  be 
indicative  of  team  performance.  For  this  study, 
sensemaking  is  assumed  to  be  dependent  on  the  currency  of 
information  provided  and  is  therefore  measured  using  time 
lag.  Box  plots  in  Figure  12  show  differences  in  time  lag 
at  the  team  level.  Team  4  is  less  variable  in  managing 
information  whereas  Team  2  shows  more  variability. 
Following  the  assumption  that  team  time  lag  can  affect 
overall  team  performance,  overall  performance  was  expected 
to  be  worse  for  Team  2  than  or  any  other  team. 


Figure  12.  Box  plots  of  Message  Time  Lag  by  Team 
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C.  RELATIONSHIPS  BETWEEN  TEAM  PERFORMANCE  2ND  PERSONALITY 

TRAITS 

The  degree  to  which  teams  consist  of  members  who  are 
similar  or  dissimilar  in  individual  characterist ics  and 
traits  may  also  explain  individual  and  team  performance. 

The  scatter  plot  in  Figure  13  compares  the  degree  to  which 
a  person  is  a  pessimist  against  their  individual 
performance  (time  lag) .  There  appears  to  be  a  positive 
relationship  between  pessimism  scores  and  time  lag.  The 
sample  correlation  r  =  .381  with  a  p-value  of  .05  for  the 
two-sided  test  of  no  correlation.  That  is  to  say,  less 
pessimistic  individuals  (those  with  lower  scores)  tend  to 
manage  information  better  than  more  pessimistic 
individuals.  Additionally,  there  appears  to  be  a  negative 
relationship  (r  =  -.391,  p-value  =.043)  between  individual 
performance  and  scores  on  extroversion.  Figure  14. 


Scatter  Plot:  Pessimism  vs  Time  Lag  (seconds) 
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Figure  13.  Pessimism  vs  Average  Time  Lag  for  Each  Individual 
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Figure  14.  Categories  of  Extroversion  vs  Individual 

Performance 

To  explore  the  hypothesis  that  team  member  similarity 
is  related  to  team  performance  as  measured  by  time  lag, 
standard  deviations  of  characterist ics  and  traits,  as 
measured  by  the  six  instruments  previously  identified,  are 
calculated  by  teams.  Standard  deviations  provide  another 
technique  for  characterizing  teams  based  on  individual 
characteristic  and  traits  of  its  team  members.  For 
instance,  in  the  category  of  neuroticism,  a  low  standard 
deviation  score  for  a  team  suggests  that  the  team  is 
similar  in  regard  to  neuroticism.  Using  this  technique,  it 
appears  that  team  similarity  in  optimism  is  strongly 
negatively  related  to  average  team  lag  (r  =  -.786,  p  -value 
=  .036)  and  that  team  similarity  in  pessimism  is  strongly 
related  to  the  number  of  lost  cells  (r  =  .821,  p-value  = 
.036) .  These  are  the  only  measures  of  diversity  related  to 
the  measures  of  team  performance.  See  Figures  15  and  16 
below  for  the  relationships  between  team  similarity  in 
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Figure  16.  Standard  Deviation  of  Scores  on  Pessimism  vs 

Number  of  Lost  Cells 

A  final  step  in  this  phase  of  the  data  analysis  is  to 
determine  which  individual  characteristic  and  trait 
measures  relate  to  performance  and  to  each  other.  A 
correlation  matrix  was  calculated  to  determine  these 
relationships  (See  Appendix  G) .  The  individual 
characteristics  and  traits  that  might  be  related  to 
performance  are  listed  below. 


A.  Individual  Performance: 

•  Pessimism  is  positively  correlated  with 

individual  performance  (time  lag,  r  =  .381, 
p-value  =  .05) 
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•  Strengths  of  Extroversion  is  negatively 
correlated  with  individual  performance  (time 
lag,  r  =  -.391,  p-value  =  .043) 

B.  Team  Performance: 

•  Optimism  is  negatively  correlated  with  team 
performance  (Average  Team  Lag  Time,  r=  - 
.786,  p-value  =  .036) 

•  Pessimism  is  positively  correlated  with 
overall  team  performance  (Number  of  Lost 
Cells,  r  =  .821,  p-value  =  .023) 

We  note  that  because  multiple  comparison  procedures  were 
not  used,  these  results  will  need  to  be  tested  in  future 
research . 

The  correlation  matrix  in  Appendix  G  also  provides 
some  insight  as  to  how  measures  of  individual 
characteristics  and  traits  are  related  to  each  other. 

There  is  evidence  to  conclude  that  each  survey  measures  a 
separate  and  distinct  individual  characterist ic  and  trait. 

D.  RESULTS  FROM  PHASE  TWO 

The  results  from  the  second  phase  of  data  analyses 
tests  whether  performance  differs  when  team  member  traits 
are  selected  by  design.  Sixteen  participants  from  the 
original  32  were  asked  to  participate  in  this  phase  of  the 
study.  Four  teams  were  constructed  so  that  all  members  on 
each  team  had  similar  agreeableness  scores  or  similar 
responses  to  uncertainty  scores: 

•  members  of  the  Team  "HA"  had  high  agreeableness; 

•  members  of  the  Team  "LA"  had  low  agreeableness; 

•  members  of  the  Team  "HURS"  had  a  high  need  to 

resolve  cognitive  uncertainty;  and 
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•  members  of  the  Team  "LURS"  had  a  low  need  to 
resolving  cognitive  uncertainty. 

Results  for  overall  performance,  as  measured  by  total 
number  of  cells  lost  due  to  ineffective  firefighting,  are: 

•  Team  HA  =  225  cells  vs  Team  LA  =  295  cells 

•  Team  LURS  =  3  cells  vs  Team  HURS  =  314  cells 
With  only  one  team  of  each  type,  we  can  not  tell  whether 
the  differences  in  performance  reflect  actual  differences. 
Thus,  Phase  Two  of  this  pilot  study  is  for  illustrative 
purposes  only. 

E.  DISCUSSION 

There  are  performance  differences  for  both  teams  and 
individuals.  For  each  trial,  the  average  lag  time  between 
messages  (information  exchange)  was  greater  for  Team  2  than 
for  any  other  team.  This  difference  indicates  that  team 
decisions  were  based  on  old  information.  Conversely,  Team 
3  practiced  information  exchange  behaviors  that  resulted  in 
team  decisions  based  on  more  current  information.  Thus, 
the  average  lag  time  between  messages  was  less.  On  overall 
performance.  Team  3  outperformed  all  other  teams  while  Team 
2  lost  the  most  cells  due  to  fire.  See  Figures  19  and  20, 
screen  shots  from  Teams  3  and  2  final  performances, 
respectively  . 
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Team  3,  Trial  1  Play 
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Figure  17.  C3Fire  Game  Screen  Shot  Team  3,  Trial  1 


Figure  1 8 . 


C3Fire  Game  Screen  Shot  Team  2, 


Trial  1 
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First,  to  determine  if  individual  and  team  performance 
differences  were  related  to  individual  characteristics  and 
traits,  we  assumed  there  would  be  differences  in  individual 
characteristics  and  traits  in  the  sample.  The  responses  to 
each  survey  were  generally  varied  and  their  distribution 
unimodal  (See  Appendix  D-F) .  If  survey  data  were  uniform, 
or  characterized  by  participants  desiring  to  load  their 
responses  by  answering  "Strongly  Agree"  to  every  question, 
the  distribution  of  responses  would  have  shown  distinct 
patterns.  Therefore,  the  assumption  that  the  sample 
consist  of  individuals  with  different  characteristics  and 
traits  is  upheld. 

Likewise,  we  expected  that  individuals  and  teams  would 
exchange  information  at  different  rates  (our  measure  of 
performance) .  As  previously  discussed,  summary  plots 
verified  that  these  differences  existed.  To  better 
understand  these  differences,  box-plots  like  the  one  shown 
in  Figure  12  were  created  to  show  how  individual 
performance  from  each  participant  relates  to  their  team 
member  performance.  Differences  in  variability  exist  as 
reflected  in  the  box-plot  of  participant  six  and  17  in 
Figure  12.  Generally,  when  comparing  performances  among 
teams,  teams  composed  of  members  who  were  less  consistent 
in  checking  messages  (information  management)  performed 
worse  than  teams  composed  of  members  who  were  more 
consistent.  Information  management  behaviors  in  Team  3  were 
more  consistent  and  outperformed  the  other  teams  in  both 
measures  of  performance:  information  currency  and  number  of 
cells  lost. 
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Individual  characteristics  and  traits  were  correlated 
with  individual  performances  to  determine  if  relationships 
exist.  From  Appendix  G,  only  two  personality  traits 
pessimism  and  extroversion  appear  to  be  related  to 
individual  performance.  The  positive  relationship  between 
pessimism  and  individual  performance  coincides  with  the 
literature  suggesting  that  feelings  of  wanting  to  give-up 
and  surrender  are  related  to  poor  performance  .  In  the  game, 
as  the  fire  became  more  out  of  controlled  and  less 
manageable,  these  individuals  would  be  expected  to  withdraw 
from  the  game  and  refuse  inputs  from  their  teammates. 
Likewise,  the  negative  relationship  between  the  degrees  of 
extroversion  (1  =  Very  Low  to  5  =  Very  High)  and  individual 
performance  also  agree  with  the  literature.  Extroverts  are 
said  to  help  facilitate  intra-team  communication  and  are 
generally  ready  to  share  information  (Bradley  &  Herbert, 
1997) .  Thus,  their  individual  time  lag  can  be  expected  to 
decrease  (i.e.,  their  performance  improves)  as  a  result  of 
being  more  extroverted. 

Also,  the  correlation  matrix  in  Appendix  H  is  color- 
coded  to  reflect  which  individual  characteristics  and 
traits  were  related  to  one  another.  The  cells  highlighted 
in  red  indicate  a  p-value  <  .01  for  testing  the  null 
hypothesis  of  correlation. 

The  second  correlation  matrix  (See  Appendix  H) 
suggests  that  diverse  teams  perform  differently  from 
homogenous  teams  on  two  of  six  individual  characterist ic 
and  trait  variables,  pessimism  and  optimism.  Time  lag 
improves  the  more  similar  team  members  are  on  their  measure 
of  optimism  (r  =  -.786,  p-value  =  .036).  These  teams 
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exhibit  quicker  information  exchanges  resulting  in  more 
current  information  (See  Figure  15) .  Conversely, 
performance  degrades  the  more  similar  team  members  are  in 
their  reports  of  pessimism.  This  finding  coincides  with 
the  belief  that  negativity  breeds  negativity. 

Results  from  phase  two  of  the  analysis  address  the 
third  question  as  to  whether  there  are  performance 
differences  when  blocking  on  various  team  member  traits  in 
team  design.  Cognitive  uncertainty  (CU)  and  the 
personality  trait  agreeableness  are  the  two  individual 
characteristics  and  traits  identified  for  this  phase  of  the 
analysis.  Attrition,  a  very  small  sample  size  to  start, 
and  the  fact  that  one  team  chose  not  to  use  any 
communication  technology  within  their  session,  prevents  any 
statistical  analysis  beyond  graphs  to  summarize  the  data. 
Figure  22  shows  that  teams  consisting  of  highly  agreeable 
members  have  lower  information  management  times  than  teams 
of  low  agreeable  members.  This  finding  is  consistent  with 
what  the  literature  informs  us  about  agreeableness  and 
information  exchange.  Teams  characterized  by  agreeable 
members  are  expected  to  exhibit  interpersonal  skills  such 
as  the  ability  to  resolve  conflict  and  communicate  openly 
in  a  manner  which  promotes  information  exchange  and 
sensemaking  (Aronoff  &  Wilson,  1985;  Neuman  &  Wright, 

1999) .  Thus,  the  time  lag  between  a  message  being  sent  and 
read  is  expected  to  be  lower  for  a  team  composed  of  non 
agreeable  members.  Average  team  time  lag  for  the  High 
Agreeable  team  was  26.4  seconds  with  a  standard  deviation 
of  25.9  where  as  the  Low  Agreeable  team  had  an  average  time 
lag  of  67.9  seconds  with  a  standard  deviation  of  56.7 
seconds . 
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VI.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

Advocates  of  NCW  claim  that  it  will  improve 
information  sharing  by  "...robustly  networking  a  force", 
thereby  improving  mission  effectiveness.  However,  this 
claim  has  not  been  tested  or  validated.  The  literature  is 
inconclusive  in  identifying  individual  characteristics  and 
traits  that  would  affect  macrocognit ive  processes  such  as 
information  sharing  and  sensemaking. 

To  guide  the  analysis  about  NCW,  the  model  of  DMSC  was 
applied  and  adapted.  This  model  provided  a  conceptual 
framework  for  describing  macrocognit ive  processes  such  as 
information  exchange  and  sensemaking.  Feedback  loops  and 
lenses  presented  in  the  model  illustrate  how  information 
can  be  omitted  or  included  in  individual  or  team  cognition. 
The  lenses  function  as  filters  which  help  characterize 
differences  in  individual  characterist ics  and  traits.  The 
feedback  loops  in  the  DMSC  are  representative  of  time  lag 
among  individuals  on  the  teams  and  time  lag  affects  on 
individual  and  overall  performance. 

Human  factors  engineering  (HFE)  is  a  discipline  that 
focuses  on  designing  systems  around  users  (i.e.,  user- 
centered  design)  and  employing  technology  that  acknowledges 
and  complements  human  characteristics  and  traits.  Although 
the  pilot  study,  by  its  very  nature,  did  not  produce 
significant  results  for  assessing  performance  differences, 
it  is  evident  that  performance  differences  can  be  explained 
by  individual  characteristics  and  traits.  There  are 
tremendous  variations  in  the  behaviors,  expectations,  and 
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mental  capabilities  of  people.  As  the  DMSC  model  depicts, 
individual  characteristics  and  traits  such  as  personality, 
cultural  adaptability,  optimism-pessimism,  and  responses  to 
uncertainty  occlude  interactions  within  teams  and 
technology.  These  differences,  if  unaccounted  for  in  team 
design,  training,  or  systems  design,  are  barriers  that 
obstruct  both  individual  and  team  performance. 

Failure  in  understanding  individual  characteristic  and 
trait  differences  within  the  DoD  population  creates 
disconnects  between  networking  technologies  and  the 
capabilities  of  the  men  and  women  expected  to  operate 
within  a  network  centric  system.  The  personnel  and 
training  domains  focus  on  identifying  these  disconnects  by 
focusing  on  the  target  audience.  The  target  audience 
includes  the  types  of  systems  used  (i.e.,  communications 
technologies  and  group  decision  support  systems)  and  key 
statistics  on  the  personnel  pool.  Assessing  users  on  the 
basis  of  personality,  cultural  adaptability,  and  optimism- 
pessimism  is  a  first  step  in  achieving  perceived  advantages 
of  distributed  teamwork.  Testing  a  variety  of  team 
compositions  of  military  personnel  will  provide  insight  as 
to  the  types  of  people  that  are  better  suited  for 
distributed  teamwork.  The  pilot  study  suggests  that 
extroverts  and  optimists  are  more  likely  to  engage  in 
information  exchanges  via  networks. 

B .  RECOMMENDATIONS 

Empirical  evidence  must  be  provided  to  facilitate  the 
acceptance  of  NCW  as  an  emerging  theory  of  warfare.  Basic 
and  applied  research  must  be  carefully  conducted  to  begin 
identifying  and  isolating  individual  characteristic  and 
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trait  variables  which  may  jeopardize  mission  effectiveness 
in  the  Information  Age.  Technology  solutions  developed 
independent  of  understanding  human  behaviors,  such  as 
macrocognition  in  teams,  invite  failure. 

C .  FOLLOW-ON  RESEARCH 

People  and  their  interaction  with  technology  are  the 
primary  and  secondary  focuses  of  NCW.  To  totally  integrate 
systems,  additional  research  must  apply  process  models  of 
human  technology  interaction  such  as  the  DMSC  to 
experimental,  field,  or  observational  settings  designed  to 
test  facets  of  NCW.  These  kinds  of  dynamic  models  provide 
extreme  possibilities  in  advancing  our  understanding  of 
which  human  factors  have  the  greatest  affect  on 
performance . 

The  C3Fire  simulation  uses  asynchronous  technology 
(i.e.,  e-mail)  for  coordination  and  communication  within 
teams.  Little  is  known  about  how  the  types  of 
technologies,  asynchronous  or  synchronous,  influence  the 
quality  of  macrocognit ive  processing  at  the  individual  or 
team  level.  Our  understanding  about  e-mail,  text-chat, 
video  teleconferencing,  or  face-to-face  communication 
systems  will  affect  the  quality  of  team  interactions  (i.e., 
communication,  planning,  and  coordination)  and  must  guide 
the  development  and  deployment  of  NCW  technologies.  Future 
research  should  include  an  examination  of  individual 
characterist ics  and  traits  in  order  to  further  understand 
which  types  of  personnel  are  more  comfortable  with 
employing  asynchronous  technologies  in  macrocognit ive 
processes.  The  literature  suggests  that  people  who  have 
high  needs  to  achieve  certainty  may  gather  and  hoard 
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information  before  contributing  or  making  a  decision.  This 
behavior  would  undermine  NCW  claims  of  networking  a 
distributed  force  to  improve  the  quality  and  amount  of 
information  sharing  to  achieve  mission  effectiveness. 
Moreover,  HFE  research  should  focus  on  varying 
communication  modalities  and  technologies  to  determine 
which  communication  modalities  enhance  information  exchange 
and  sensemaking,  making  them  better  at  facilitating 
distributed  teamwork. 

Research  about  human  performance  has  traditionally 
used  civilian  undergraduate  students.  However,  military 
transformation  implies  understanding  the  current  population 
of  users,  and  redeveloping  tactics,  techniques,  and 
procedures  that  better  enable  them  to  achieve  mission 
effectiveness.  Critical  information  in  determining 
military  personnel  requirements  needs  to  be  gathered. 
Therefore,  human  performance  researchers  must  redefine 
their  target  populations  and  exploit  service  men  and  women 
at  various  military  schools  . 

The  first  tenet  of  NCW  postulates  that  if  a  force  is 
robustly  networked,  then  they  would  share  information  and 
that  this  information  exchange  would  lead  to  improve 
mission  effectiveness.  There  are  other  human  factors  such 
as  organizational  structure,  cross  planning  and  execution 
between  military  versus  non-military  organizations,  trust, 
and  team  cohesion,  which  affect  team  performance. 

Objective  and  qualitative  measures  of  individual  and  team 
performance  in  a  network  environment  need  to  be 
established.  The  results  of  individual  and  team  measures 
could  then  be  used  to  maximize  human-system  performance. 
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APPENDIX  A.  GLOSSARY  AND  ACRONYMS 


AACS  Ability  to  Achieve  Cognitive  Structure, 

the  extent  to  which  an  individual  is 
able  to  avoid  information  that  either 
cannot  be  categorized  or  clashes  with 
their  existing  knowledge  and/or  ability 
to  organize  their  knowledge  to  fit  an 
already  existing  cognitive  structure 
(Bar-Tal ,  1994,  p.  46) 

AWACS-AEDGE  Airborne  Warning  and  Control  System  - 
Agent  Enabled  Decision  Guide 
Environment 

CA  Conflict  Avoidance 

C2  Command  and  Control 

DMSC  Dynamic  Model  of  Situated  Cognition 

HSI  Human  Systems  Integration 

HSIL  Human  Systems  Integration  Laboratory 

NCS  Need  for  Cognitive  Structure,  the 

desire  for  clear  and  firm  knowledge 
concerning  a  given  topic  as  opposed  to 
ambiguity  doubt,  or  confusion 
(Kruglanski,  1989,  as  cited  in  Bar-Tal, 
1994,  p.  46) 

NCW  Network  Centric  Warfare,  the  conduct  of 

military  operations  using  networked 
information  systems  to  generate  a 
flexible  and  agile  military  force  that 
acts  under  a  common  commander's  intent, 
independent  of  the  geographic  or 
organizational  disposition  of  the 
individual  elements,  and  in  which  the 
focus  of  the  warfighter  is  broadened 
away  from  individual,  unit  or  platform 
concerns  to  give  primacy  to  the  mission 
and  responsibilities  of  the  team,  task 
group  or  coalition.  (Fewell  &  Hazen, 
2003,  p.  39) 

NEO-FFI  NEO  -  Five-Factor  Inventory 
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OP  I 


Optimism 


Pessimism  Instrument 


Optimism 

Pessimism 

SABRE 

Sensemaking 

URS 

Virtual  Team 


A  disposition  inclining  one  to  positive 
expectations 

A  disposition  inclining  one  to  negative 
expectations 

Situation  Authorable  Behavior  Research 
Environment 

A  behavior  that  is  both  internal  (i.e., 
cognitive)  and  external  (i.e. 
procedural),  which  allows  the 
individual  to  construct  and  design 
their  movement  through  time-space 
(Dervin,  1983,  p.3) 

Uncertainty  Response  Scale 

a  collection  of  individuals  who 
are  interdependent  in  their  tasks,  who 
share  responsibility  for  outcomes,  who 
see  themselves  and  who  are  seen  by 
others  as  an  intact  social  entity 
embedded  in  one  or  more  larger  social 
systems,  and  who  manage  their 
relationship  across  organizational 
boundaries.  (Powell,  Piccoli,  and  Ives 
,2004,  p.  241) 
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APPENDIX  B.  NCW  DEPENDENT  VARIABLE 


A.  INFORMATION  EXCHANGE  (CURRENCY,  FROM  EVIDENCED  BASED 
RESEARCH,  2004,  P.37) 


Attribute 

Metrics 

Objective  Measures 

Measures  quality  in  reference  to  criteria  that  are  independent  of  the  situation 

Correctness 

Correspondence  with  ground  truth  correlation  coefficient  (0=  no  convergence,  l=full 
convergence  between  individuals  awareness  and  ground  truth) 

Consistency 

Degree  of  ‘deviation1  from  awareness  gained  from  previous  time  period 

Currency 

Time  lag  of  awareness 

Precision 

Level  of  granularity  of  awareness 

Fitness  for  Use 

Measures 

Measures  quality  in  reference  to  criteria  that  are  determined  by  the  situation 

Completeness 

Percentage  of  ground  truth  picture  included  in  awareness 

Accuracy 

Degree  to  which  precision  matches  what  is  needed  (0=no  match,  10=high  degree  of  matching 
between  precision  level  needed  and  available) 

Relevance 

Proportion  of  awareness  that  is  related  to  task  at  hand 

Timeliness 

Degree  to  which  currency  matches  what  is  needed  (0=no  match,  10=high  degree  of  matching 
between  currency  level  needed  and  available) 

Uncertainty 

Confidence  level  (0%  =uncertain,  100%=  certain)  or  confidence  interval  (95%,  90%,  etc.)  of 
awareness 

Time 

of 

Event 

«  | 

Time 

Event 

Posted 

I 

Time 

Information 

Received 

at 

Command  Ctr 

1 

Time 
Event 
Noticed 
by  Staff 

1 

Time 

Event 

Significance 
Projected 
by  Staff 

1 

Time 

Event 

Response 

Declared 

by 

Commander 

_ 

"^"event 

Tpost 

"^receive 

- W' 

^"awareness  ^"understanding  T^e^g^n 

V _ 

- V - 

Time  lag  of  awareness 

_ V 
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APPENDIX  C.  POINTS  OF  CONTACT 


A.  NAVAL  POSTGRADUATE  SCHOOL 

Nita  Lewis  Miller,  Ph.D. 

Director,  Human  Systems  Integration  Program 
Lyn  R.  Whitaker,  Ph.D. 

Associate  Professor  of  Operations  Research 
Lawrence  G.  Shattuck,  Ph.D. 

Co-Director,  Human  Systems  Integration  Program 
Senior  Lecturer 

B .  C3FIRE 

Rego  Granlund,  Ph.D. 

Developer  of  C3Fire 

Professor,  Linkoping  Institute  of  Technology 
Kip  Smith,  Ph.D. 

Professor,  Linkoping  Institute  of  Technology 
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APPENDIX  D.  SURVEY  RESULTS 


Personality  results  measured  using  the  NEO-FFI 


Extroverts 


Openness 


Very  Low  Average  High  Very 
Low  High 


Agreeableness 


Neurotics 


Conscientiousness 


15 


Low  High 

Bin 
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APPENDIX  E.  SURVEY  RESULTS 


Optimism  and  Pessimism  results  using  the  OPI . 


Responses  to  the  Optimism  Pessimism  Instrument  (OPI) 


Responses  to  Uncertainty  using  the  URS 


Conflict  Avoidance  Responses 


Conflict  Avoidance  (CA)  Scores 


Participant 
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APPENDIX  F.  SURVEY  RESULTS 


Ability  to  Achieve  Cognitive  Structure  Responses 


AACS  Total 


Participants 


AACS  Total 


Need  for  Cognitive  Structure  Responses 


NCS  Total 


Particiants 


-♦ — NCS  total/person 
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APPENDIX  G.  CORRELATION  MATRIX  OF  INDIVIDUAL 
CHARACTERISTICS  AND  TRAITS  WITH  TEAM  PERFORMANCE 
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APPENDIX  H.  CORRELATION  MATRIX  OF  TEAM 
CHARACTERISTICS  (DEGREE  OF  SIMILARITY)  WITH 

PERFORMANCE 


Team  Average 

Emotion 

Desire  fo 

Cognitive 

Conflict 

Number 

Time  Lag  (in 

Optimisim( 

Pessimisn 

Extroversio 

Openness! 

Agreeable) 

Neuroticism 

Conscientiousness 

Uncertainty 

Chanqe! 

Uncertainty 

Avoidance 

of  Lost 

sec) 

similiarity) 

(similarity) 

n(similarity) 

similarity) 

similarity) 

(similarity) 

(similarity) 

similarity) 

similarity) 

(similarity) 

(similarity) 

Cells 

Spearman's  Team_Aver  Correlation 
rho  age  Time  Coefficient 

1.000 

-0.78f 

-0.321 

■0.179 

-0.750 

-0.429 

-0.393 

-0.286 

0.286 

0.107 

0.250 

-0.429 

•0.393 

Lag  (in  sec)  Sig.  (2-tailed) 

0.036 

0.482 

0.702 

0.052 

0.337 

0.38: 

0.535 

0.535 

0.819 

0.585 

0.337 

0.383 

N 

7 

7 

7 

7 

7 

7 

7 

7 

Optimisim  Correlation 
(similarity)  Coefficient 

-0.786 

1.006 

0.464 

0.107 

0.464 

0.750 

0.536 

0.500 

-0.143 

-0.250 

-0.607 

0.393 

0.607 

Sig.  (2-tailed) 

0.036 

0.29' 

0.81! 

0.29' 

0.052 

0.216 

0.253 

0.76C 

0.585 

0.146 

0.385 

0.148 

N 

* 

7 

7 

7 

7 

7 

7 

7 

Pessimism  Correlation 

-0.321 

0.4& 

1.000 

0.607 

0.071 

0.036 

0.857 

0.536 

-0.036 

•0.071 

-0.393 

0.357 

0.821 

Sig.  (2-tailed) 

0.482 

0.29' 

. 

0.146 

0.875 

0.939 

0.014 

0.215 

0.935 

0.875 

0.383 

0.432 

0.023 

N 

7 

7 

7 

7 

7 

7 

7 

7 

Extroversio  Correlation 

-0.179 

0.107 

0.607 

1.000 

0.321 

-0.286 

0.679 

0.286 

0.393 

0.429 

0.357 

0.321 

0.714 

Sig.  (2-tailed) 

0.702 

0.815 

0.146 

. 

0.482 

0.535 

0.09' 

0.535 

0.385 

0.337 

0.432 

0.482 

0.071 

N 

7 

7 

7 

7 

7 

7 

7 

7 

Openness  Correlation 

-0.750 

0.4& 

0.071 

0.321 

1.000 

0.214 

0.214 

-0.107 

0.321 

0.286 

0.321 

0.679 

0.393 

Sig.  (2-tailed) 

0.052 

0.29' 

0.875 

0.482 

0.645 

0.645 

0.819 

0.482 

0.535 

0.482 

0.09' 

0.383 

N 

7 

7 

7 

7 

7 

7 

7 

7 

Agreeable  Correlation 

-0.429 

0.756 

0.036 

-0.286 

0.214 

1.000 

0.321 

-0.036 

0.036 

-0.643 

-0.464 

-0.071 

0.143 

Sig.  (2-tailed) 

0.337 

0.052 

0.936 

0.535 

0.645 

0.482 

0.939 

0.935 

0.119 

0.29' 

0.875 

0.760 

N 

7 

7 

7 

7 

7 

7 

7 

7 

Neuroticism  Correlation 

-0.393 

0.536 

0.857 

0.679 

0.214 

0.321 

1.000 

0.250 

0.179 

-0.286 

-0.179 

0.143 

0.750 

Sig.  (2-tailed) 

0.383 

0.216 

0.01' 

0.09' 

0.645 

0.482 

0.589 

0.702 

0.535 

0.702 

0.76C 

0.052 

N 

7 

7 

7 

7 

7 

7 

7 

7 

Conscientio  Correlation 
usness  Coefficient 

-0.286 

0.506 

0.536 

0.286 

-0.107 

-0.036 

0.250 

1.000 

-0.429 

0.286 

-0.643 

0.321 

0.571 

(similarity)  Sig.  (2-tailed) 

0.535 

0.25: 

0.216 

0.53! 

0.815 

0.939 

0.585 

. 

0.337 

0.536 

0.115 

0.482 

0.180 

N 

7 

7 

7 

7 

7 

7 

7 

7 

Emotion  Correlation 
Uncertainty  Coefficient 

0.286 

-0.14: 

-0.036 

0.393 

0.321 

0.036 

0.179 

-0.429 

1.000 

0.321 

0.607 

0.321 

0.321 

(similarity)  Sig.  (2-tailed) 

0.535 

0.766 

0.935 

0.38: 

0.485 

0.939 

0.705 

0.337 

0.482 

0.145 

0.482 

0.482 

N 

7 

7 

7 

7 

7 

7 

7 

7 

Desire_for_  Correlation 

0.107 

-0.256 

-0.071 

0.429 

0.286 

-0.643 

-0.286 

0.286 

0.321 

1.000 

0.429 

0.607 

0.286 

(similarity)  Sig.  (2-tailed) 

0.819 

0.585 

0.875 

0.337 

0.535 

0.119 

0.535 

0.535 

0.482 

0.337 

0.145 

0.535 

N 

7 

7 

7 

7 

7 

7 

7 

7 

Cognitive_  Correlation 

0.250 

-0.607 

-0.393 

0.357 

0.321 

-0.464 

-0.179 

-0.643 

0.607 

0.429 

1.000 

0.000 

-0.214 

(similarity)  Sig.  (2-tailed) 

0.589 

0.146 

0.385 

0.432 

0.482 

0.294 

0.702 

0.119 

0.145 

0.337 

1.00C 

0.645 

N 

7 

7 

7 

7 

7 

7 

7 

7 

Conflict  Correlation 

-0.429 

0.39: 

0.357 

0.321 

0.679 

-0.071 

0.143 

0.321 

0.321 

0.607 

0.000 

1.000 

0.679 

(similarity)  Sig.  (2-tailed) 

0.337 

0.38: 

0.432 

0.482 

0.09' 

0.879 

0.76( 

0.482 

0.482 

0.148 

1.000 

. 

0.094 

N 

7 

7 

7 

7 

7 

7 

7 

7 

Number  of  Correlation 

-0.393 

0.607 

0.821 

0.714 

0.393 

0.143 

0.750 

0.571 

0.321 

0.286 

-0.214 

0.679 

1.000 

Sig.  (2-tailed) 

0.383 

0.148 

0.023 

0.071 

0.385 

0.760 

0.052 

0.180 

0.482 

0.535 

0.645 

0.09' 

N 

7 

7 

7 

7 

7 

7 

7 

7 
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